Learning Human Search Behavior from Egocentric Visual Inputs

"Looking for things" is a mundane but critical task we repeatedly carry on in our daily life. We introduce method to develop human character capable of searching randomly located target object detailed 3D scene using its locomotion capability and egocentric vision perception represented as RGBD images. By depriving the privileged information from character, it forced move look around simultaneously account restricted sensing capability, resulting natural navigation search behaviors. Our consists two components: 1) control policy based an abstract model, 2) online replanning module synthesizing kinematic motion trajectories planned by policy. demonstrate that combined techniques enable effectively find often occluded household items indoor environments. The same can be applied different full-body characters without need retraining. evaluate quantitatively testing generated scenarios. work first step toward creating intelligent virtual agents with humanlike behaviors driven onboard sensors, paving road future robotic applications.